The present invention relates to transactional recoverability, and in particular, to transactional recoverability in a distributed computing environment.
Unless otherwise indicated herein, the approaches described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.
Within many business enterprises today, enterprise applications are leveraging multiple distributed resources which exist within or outside of the enterprise. These resources can be databases, messaging systems, or even applications offering services to the outer world.
In many scenarios it is desirable that updates across such systems are done in a transactional fashion. This basically means the updates have to comply to the ACID criteria (atomicity, consistency, isolation and durability). Atomicity means that changes in the distributed resources are performed at all resources or at none. Consistency means that when the transaction ends, the integrity constraints between these resources must be valid. Isolation means that no operation outside of the transaction should see an intermediate state. Durability means that if the transaction is successfully finished, it will persist and not be undone.
These criteria are today met by nearly all resources which are common within enterprises. A database management system (DBMS) which is asked to withdraw money from one account and to deposit it into another one in a single transaction can be expected to do this flawlessly according to the ACID rules outlined above. The DBMS itself is in full control of the transaction—it is as well the transaction manager and the resource being managed by the transaction manager.
The situation changes fundamentally if a transaction has to span multiple resources. The application server becomes the transaction manager and all resources becomes subordinates of this transaction manager. Because the overall state of the transaction is now distributed over all involved resource managers there will be some amount of time during which the overall transaction is inconsistent.
This inconsistency is not something which can be avoided—it lies in the very nature of distributed systems. So if the system performing the transactional work or parts of it crash the inconsistency will endure. An example of such a situation could be that the application server crashing after one resource has done its changes but the second one has not.
To overcome these issues, transaction recovery strategies may be used. If the systems involved in the transaction become available again, a transaction recovery functionality within the application server remembers what was promised to the application triggering the transaction. The application server checks with the resources about the outcome of their part of the transaction and tries to achieve a consistent state. If the application server is unable to achieve a consistent state it informs the operator about this inconsistency between the involved resources—so at least the problem is identified.
Without transaction recoverability functionality, customer data most likely will become corrupt if distributed transactions are used. That is the situation as it is with the current architecture of the Java™ application server. As described in the book Java Transaction Processing: “Some application servers (today this applies mostly to experimental or Open Source projects rather than enterprise-quality servers) do not provide recovery capabilities and should not be used for applications that interact with multiple recoverable resources that require transactions.” See Mark Little et al., Java Transaction Processing (Prentice Hall PTR 2004).
Implementing transaction recovery is a fundamental change for an application server. New recovery functions may be created and used. In addition, the way resources are implemented may be changed. This may require further modifications to other systems, such as the Java™ Messaging System provider within the server, the Transaction Manager, the JDBC (Java™ Database Connectivity) DB (database) Pool Service, and the Java Connector Container.
If an application server and together with it the transaction manager (which was a part of the application server) both crash, the resources which were involved in the transaction are left alone. This loneliness of resources is risky as there is still a transaction pending: After some time the resource will make a decision to commit or to roll back the part of the transaction it was involved in. This decision might be according to the original intent of the Transaction Manager or not—it is a so called heuristic decision. So whenever possible heuristic decisions must be avoided.
A distributed computing environment generally includes a number of tiers, with each tier having one or more computing devices. For example, a two-tier architecture may include a client tier and a server tier, with multiple client devices and multiple server devices in each tier. A three-tier architecture may include a client tier, an application server tier, and a database server tier (again with multiple computing devices in each tier).
The application server tier may also be referred to as an application server cluster. An application server cluster may consist of many server nodes which can perform transactions across distributed resources. The application servers may implement a computing environment such as Java™ Enterprise Edition, Microsoft .NET™ environment, etc.
The database server tier may also be referred to as a resource tier. An application which lives on an application server can use different resources (of the resource tier) in a transactional manner. Examples of resources are database systems, messaging systems, and enterprise information systems.
The part of the application server which controls transactions is called Transaction Manager (TM). The semantics the TM uses to orchestrate transactions across these distributed resources may be described by the DTP (Distributed Transaction Processing) XA Specification from the Open Group.
The TM manages transactions on behalf of the application living on the server. This means it is responsible for keeping all resources in a consistent state. Because the overall state of the transaction is distributed across all involved resource managers there will be some amount of time during which the overall transaction is inconsistent.
This inconsistency is not something which can be avoided—it lies in the very nature of distributed systems. So if the system performing the transactional work (or parts of it) crash, the inconsistency will endure. An example of such a situation could be that the application server node crashes after one resource has done its changes but the second resource has not. If one or more of the application server nodes crashes, the distributed transactions may remain in an incomplete state.
As every participant of such a transaction can crash it is necessary that each participant writes it decision to commit or rollback to a persistent store. This store is called the Transaction Log (TLOG).
Transaction processing may be done according to the DTP XA Specification of the Open Group. This specification divides the completion of a transaction into two phases: a prepare phase, and a commit phase.
A transaction completion sequence with two participants is described below. Resource One is connected to the application server with Resource Adapter One, Resource Two with Resource Adapter Two. Each participant has its own TLOG. Thus, overall there are three TLOGs: one which is maintained by the TM, and two more which are controlled by their respective resource managers.
The transaction completion process is as follows:
1. The application asks the application server to commit the transaction.
2. The application server informs the TM that the transaction should be completed.
3. The TM initiates the first phase of the transaction completion: It asks the first resource manager (RM) to prepare the transaction.
4. We assume that the first RM is able to commit and writes this intent to its own transaction log. The first RM returns from the prepare call signaling the TM success.
5. The same happens for the second RM.
6. As we assume the TM received only successful prepare return codes it decides to commit the complete transaction. The TM writes its decision to commit in his own TLOG before it reaches out to the RMs. After the write operation, the TM will try to commit this transaction even if it crashes immediately after the write.
7. The first RM is asked to commit. Normally it should deliver on the promise it gave to the TM during prepare time and commit without any problem.
8. The second RM is asked to commit.
A problem before any prepare call is made is not critical. After some time each participant will abandon the transaction which leads finally again to a consistent state.
More critical are problems after prepare. An example could be an application server crash after step 7 but before step 8. This would lead to Resource Manager One having committed its transaction branch already and Resource Manager Two still staying in the prepare state.
If this situation endures, Resource Manager Two will (after some time) decide itself about the outcome of its transaction branch. It will make a heuristic decision. Many existing resource managers will decide to roll back their branch. If this should happen, the overall transaction is inconsistent. The DTP XA specification calls this a “heuristic mixed transaction outcome”.
Furthermore, the completion process can take a while, especially if resource managers are slow. Imagine a slow RM One: If there is a failure during the second phase of the 2PC (2 phase commit) protocol, the overall transaction is inconsistent. In many current implementations of the application server, this inconsistency will go unnoticed so chances are pretty high that customer data can become corrupted.
Existing application servers then attempt recovery of these distributed transactions using a recovery service.